Interactive Occlusion and Collision of Real and Virtual Objects in Augmented Reality

We present several techniques for interactively performing occlusion and collision detection between static real objects and dynamic virtual objects in augmented reality. Computer vision algorithms are used to acquire data that model aspects of the real world. Either geometric models may be registered to real objects, or a depth map of the real scene may be extracted with computer vision algorithms. The computer vision-derived data are mapped into algorithms that exploit the power of graphics workstations, in order to interactively produce new effects in augmented reality. By combining live video from a calibrated camera with real-time renderings of the real-world data from graphics hardware, dynamic virtual objects occlude and are occluded by static real objects. As a virtual object is interactively manipulated collisions with real objects are detected, and the motion of the virtual object is constrained. Simulated gravity may then be produced by automatically moving the virtual object in the direction of a gravity vector until it encounters a collision with a real object.